Skip to content

Owner-only /_api/logs for any container; ship-fix keeps the pod's logs private - #156

Merged
amiller merged 1 commit into
mainfrom
feat/owner-container-logs
Nov 7, 2026
Merged

amiller merged 1 commit into
mainfrom
feat/owner-container-logs

Conversation

@amiller

@amiller amiller commented Nov 7, 2026

Copy link
Copy Markdown
Owner

Phala's public_logs flag is all or nothing. With it on, anyone who knows the CVM's instance id can download every container's full log, which for the daemon means its complete access log. With it off, nobody can read the logs, the owner included. phala deploy also defaults the flag to true, so ship-fix.sh has been re-publishing the pod's logs on every deploy.

The daemon already holds the Docker socket and the owner token, so it can serve the logs itself.

Change

  • GET /_api/logs lists every container on the CVM (name, state, status, image).
  • GET /_api/logs/<container> streams that container's stdout and stderr with timestamps. tail defaults to all, and since/until take unix seconds. The response is streamed and demuxed frame by frame, since the pod's daemon log alone is several hundred MB.
  • Both routes are owner-only. A scoped token gets 403, as with /_api/stats and /_api/tokens.
  • The container name goes into a Docker API path, so it is restricted to Docker's name alphabet. dstack's guest agent needed the same check (fix(guest-agent): reject unsafe container names in the logs route (backport #1442) Dstack-TEE/dstack#1443). tail, since and until are checked the same way.
  • /_api/projects/<name>/logs stays as it is. It is capped at 2000 lines and only reaches a project's own container. The new route also reaches the daemon, the shared runtimes and the compose siblings such as the ingress.

ship-fix.sh:

  • The pod target passes --no-public-logs.
  • Before building, it compares the local compose with the live one (phala cvms get --json), ignoring the daemon image, and refuses to deploy if anything else differs. The script only swaps the daemon digest, so any other drift in the local file was being shipped silently. The prod7 compose in hermes-agent is currently behind the pod: older daemon, no PROJECT_HOSTS, and one ingress domain instead of two. Shipping it would break private-agent's domains, so it has to be refreshed from the live compose before the next pod deploy.

Upstream, Dstack-TEE/dstack#1469 (merged 2026-10-28) gates the guest agent's log route on a signed token in addition to public_logs. That is the general fix, but it is not in any OS image Phala offers yet (newest is dstack-0.6.1).

Tests

  • proxy/test_owner_logs.py, 7 tests:
    • owner reads the full history, and the query parameters reach Docker unchanged
    • listing
    • a scoped token is refused on both routes
    • bad names (including an encoded ../), a bad tail or timestamp, and a smuggled &follow=true all get 400
    • a missing container gets 404
    • stream_logs runs against a fake Docker Engine on a unix socket. It demuxes stdout and stderr frames split across writes, including a 70 KB frame. TTY containers pass through raw, and a missing container raises before anything is written.
  • Without the change, the 7 new tests fail and the 23 existing ones pass. With it, all 30 pass (python3 -m pytest proxy/).
  • The drift check, run on its own against oauth3-prod7: it refuses the current local docker-compose.prod7.yaml and names tee-daemon and cpanel-ingress as differing. It accepts the live compose.

Not deployed. Rolling it out to the pod means one CVM upgrade, which also turns public_logs off: refresh the local compose from the live one, then ./ship-fix.sh pod.

…s private

GET /_api/logs lists every container on the CVM and GET /_api/logs/<name>
streams one container's full stdout/stderr (tail=all by default, since/until
in unix seconds). Both take the owner token only. This replaces Phala's
public_logs endpoint, which serves the same logs to anyone.

ship-fix.sh passes --no-public-logs for the pod (phala deploy defaults it to
true), and refuses to deploy when the local compose differs from the live one
in anything but the daemon image.
@amiller
amiller merged commit d4f7417 into main Nov 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant